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Summary 


Researchers have linked positive student outcomes, including student achievement, to 
high-quality school leadership (Grissom & Loeb, 2011). Ensuring the quality of school 
leaders is a priority across the Regional Educational Laboratory (REL) Southeast Region. 
In Florida the Department of Education is revisiting state and district policies regulating 
how principals are trained, certified, and evaluated. 

To support these reform efforts, the Florida Department of Education partnered with REL 
Southeast to conduct a systematic review of research to illuminate the potential relation- 
ships between principal characteristics and student achievement. Each document selected 
for inclusion was systematically reviewed and the key findings were synthesized to answer 
the question: What is known about the relationships between principal characteristics and 
student achievement? 

This systematic review examined 52 empirical studies published between 2001 and 2012 
on the relationships between principal characteristics and student achievement in the 
United States. 

Key findings include: 

• The only randomized controlled trial study that addressed the relationships 
between principal characteristics and student achievement found that grade 8 
students randomly assigned to have one-to-one conversations with the principal 
scored higher on the state English language arts test than students who did not 
have such conversations (Silva, White, & Yoshida, 2011). 

• All other studies provided correlational or descriptive information about the rela- 
tionships between principal characteristics (including precursors, behaviors, and 
leadership styles) and student achievement. 
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Why this review? 


Due in part to research that has linked the quality of school leadership to improved student 
outcomes, federal and state educators across the country have expressed increased interest 
in improving the quality of school leaders as a way to improve student achievement. In 
Florida the Department of Education is revising and updating its policies related to princi- 
pal training, certification, and evaluation. To support this effort, the Florida Department 
of Education partnered with Regional Educational Laboratory (REL) Southeast to conduct 
a systematic review of research regarding what is known about the relationships between 
principal characteristics and student achievement to help inform their decisionmaking. 

The idea that effective school leadership contributes to student achievement is neither 
new nor controversial (Murphy, Elliott, Goldring, & Porter, 2006). A growing research 
base examines principals’ influence on student academic achievement. However, much of 
the earlier research found that the relationships are statistically weak (see box 1 for an 
explanation of statistical relationships and effect sizes). For example, a meta-analysis of 
international research conducted from 1986 to 1996 found that the average correlation 
between educational leadership and student achievement was negligible, less than r = .10 
(Witziers, Bosker, & Kruger, 2003). A more comprehensive meta-analysis of more than 30 
years of research through 2001 found a somewhat stronger correlation of r = .25 (Marzano, 
Waters, & McNulty, 2005 1 ; Waters, Marzano, & McNulty, 2003). While other researchers 
found more significant correlations between specific leadership responsibilities (Marzano 
et al., 2005) or leadership styles (Robinson, Llloyd, & Rowe, 2008 2 ) and student achieve- 
ment, there has been little empirical research on the relationships between precursors 3 
(that is, principals’ experience, knowledge, personal traits, values, and beliefs) to their 
leadership behaviors and student achievement (see appendix A for results from previous 
reviews and meta-analyses and box 2 for definitions of leadership styles). 

Historically, the principal’s role has been to focus on school organization, performance, and 
staff morale (Crow & Glascock, 1995; Osterman, Crow, & Rosen, 1997). Thus research- 
ers studied those roles. Moreover, because principals defined themselves in organizational 
roles, it is possible they did not reach their potential for increasing student achievement. 

With the recent focus on student achievement, the principal’s role has shifted to instruc- 
tional leadership (Blase & Blase, 1998; Brookover & Lezotte, 1982). It is possible that prin- 
cipals have stronger impacts on student achievement than previous research suggested, 
and that those impacts may be increasing as principals adapt to changing expectations. 
Thus, this review focuses on recent studies with explicit connections between principal 
characteristics and student outcomes. 
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Box 1. Interpreting effect sizes 

Effect sizes are presented throughout the report. An effect size is a number that reflects the 
strength of a relationship between two variables. In this context, effect sizes express the 
magnitude of the relationship between a principal’s leadership characteristics and student 
achievement (Cooper, Hedges, & Valentine, 2009). While a variety of effect sizes can be used, 
for consistency this study reports effect sizes as r (see appendix C for other types of effect 
sizes). The r effect size ranges from -.99 to +.99. A positive value means that an increase in 
the principal behavior is related to an increase in student achievement and that a decrease in 
the principal behavior is related to a decrease in student achievement. A negative value means 
than an increase in principal behavior is related to a decrease in student achievement and that 
a decrease in principal behavior is related to an increase in student achievement. The size of 
the relationship using rcan be interpreted as follows (Cohen, 1988; Rosenthal, 1996): 

• + .00-.09, no or negligible effect. 

• + .10-29, small effect. 

• + .30-49, medium effect. 

• + .50-69, large effect. 

• ± .70-. 99, very large effect. 

If it is not possible to calculate an effect size as r, standardized regression coefficients 
((3) are reported. Like r, these effect sizes can be positive or negative; a positive (3 means that 
an increase in principal behavior is related to an increase in student achievement. Unlike r, 
(3 can exceed ± 1. (5 is interpreted as the standard deviation change in student achievement 
associated with a one standard deviation increase in principal behavior. For example, a (3 of 
.85 means that a one standard deviation increase in principal behavior is associated with a 
.85 standard deviation increase in student achievement. 


Box 2. Definitions of key terms 

Collegial leadership. Behavior that is supportive of teachers and collegiality between school 
leaders and teachers. 

Distributive leadership. A leadership approach with a collaborative focus where leadership is 
shared and practiced by all members of the school, not just the principal. 

Instructional leader. Traditionally assumed to be the principal or school leader, though more 
recently the concept has broadened to include other key staff such as lead teachers. 

Instructional leadership or pedagogical leadership. Approach whereby the leader helps foster 
a learning climate free of disruption, a system of clear teaching objectives, and high teacher 
expectations for students. Elements include principal leadership, clear mission, teaching 
expectations, and opportunities to learn. 

Learning-centered framework. A framework whereby experience, knowledge, personal traits, 
values, and beliefs are precursors to a principal’s leadership behaviors, which in turn influ- 
ence the overall school experience (for example, standards, curriculum, culture) and specific 
classroom experiences (for example, teacher practices) of the student, which affect student 
success (Murphy et al., 2006). 


(continued) 
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Box 2. Definitions of key terms (continued) 


Learning-directed leadership Distributed or collaborative leadership combined with the devel- 
opment of school capacity to support teaching and learning. 

Nonacademic outcomes. Student outcomes such as self-concept or participation in school 
activities. 

Organizational leader. The leader of the organization, in this case, the school. The role of the 
organizational leader has historically focused on overall organization, performance, and staff 
morale. 


Precursors. Principals’ experience, knowledge, personal traits, values, and beliefs that influ- 
ence their leadership behaviors. In this review precursors are defined as conditions that shape 
principal behavior, as conceptualized by Murphy et al. (2006). However, the precursor catego- 
ries in this review are not directly aligned with Murphy et al.’s definitions. 

Student achievement. Academic achievement, typically measured through assessments of 
math and English language arts. 

Transformational leadership. Approach whereby the leader inspires others with a vision that 
energizes them and encourages them to work collaboratively toward a common goal. Elements 
include individualized consideration, intellectual stimulation, inspirational motivation, and ide- 
alized influence. 


This review 
included studies 
conducted in all 50 
states, at least 329 
school districts, 
and at least 
8,363 schools 


How the review was conducted 


The first step in this systematic review of the literature was to define a range of poten- 
tial principal characteristics that might influence student achievement. For this review 
“principal characteristics” was defined broadly as anything a principal has experienced (for 
example, training, job experience), does (for example, specific actions such as mentoring 
students or teachers), or believes (for example, expectations of students and staff). This 
definition was used to develop the search for research linking principal characteristics to 
student achievement. The studies were later grouped into three categories of principal 
characteristics: precursors, behaviors, and leadership styles (see box 3 for the review meth- 
odology, appendix B for guiding leadership frameworks, and appendix C for the review 
protocol). 

This review included studies conducted in all 50 states, at least 329 school districts, and at 
least 8,363 schools. The review team reviewed 803 unique studies that employed a variety 
of quantitative and qualitative analytic techniques. Of these studies, 52 met the screening 
criteria: 39 quantitative studies, 11 qualitative studies, and 2 mixed method studies (see 
figure D1 in appendix D for screening criteria and study characteristics). 4 

What the systematic review found 


The only experimental study found that a principal talking one-on-one with students 
before a test increased their test scores. Of the other 51 studies, 38 quantitative and 
2 mixed method studies showed various correlations between principal characteristics and 
student achievement. However, correlations do not indicate causality. 
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Box 3. Methodology 


One research question guided this systematic review: What is known about the relationships 
between principal characteristics and student achievement? 

To answer the question, the review team conducted a comprehensive search for research 
on principal characteristics and student achievement using a protocol of standardized proce- 
dures (see appendix C). The protocol was intentionally broad to capture a wide range of study 
types (qualitative case studies and experimentally designed quantitative studies). Review pro- 
cedures included setting criteria for inclusion and exclusion, using multiple methods to search 
for potential literature, conducting multiphase screening for inclusion and exclusion by multiple 
trained reviewers, systematically coding the focus of each included study, and tabulating the 
results of the included studies. 

Both quantitative and qualitative research were included in this systematic review. To be 
considered for inclusion, studies had to: 

• Be an empirical study in a K-12 education setting in the United States. 

• Have been peer reviewed (for example, peer-reviewed journal articles, Institute of Educa- 
tion Sciences publications). 

• Have been published between 2001 and 2012. 

• Include operationally defined, measurable principal characteristics. 

• Include at least one measure of student achievement. 

• Specifically analyze the relationship between the principal characteristic and student 
achievement. 

This report highlights principal characteristics identified in the quantitative literature and 
supported by qualitative research that describes the application of the characteristics. 

Where applicable, effect sizes were calculated using statistics reported in the studies. 
Since the relationships between principal characteristics and student achievement are of inter- 
est, the effect size is reported as r to estimate the magnitude of the relationship between 
those two variables (see box 1 and appendix C). 

The relationships between principal characteristics and student achievement were more 
difficult to ascertain in studies using qualitative analysis. Therefore, qualitative studies were 
included only if an explicit description of student achievement was part of the selection criteria 
for the study. For example, a qualitative study may have compared various characteristics 
of the principals from the top- and bottom-performing schools based on statewide student 
achievement tests to determine whether there was a qualitative difference between the two 
groups. 


This report 
highlights principal 
characteristics 
identified in the 
quantitative 
literature and 
supported by 
qualitative 
research that 
describes the 
application of the 
characteristics 


One experimental study found that a principal talking with students about their performance prior to 
a statewide test increased test scores 

There was only one experimental study among the quantitative studies (Silva et al, 2011). 
Experimental studies are the only type of studies that can provide credible evidence on 
whether an intervention is effective. 

Silva et al. (2011) examined the effect of principals having one-on-one conversations with 
grade 8 students regarding their achievement on an annual state English language arts 
test. All grade 8 students in the district who were not proficient on the 2008 grade 7 test 
were randomly assigned one of two interventions. 
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Experimental group students engaged in individual achievement-based conversations with 
the principal a month before the 2009 grade 8 test. Each student engaged in two 15-minute 
conversations during which the principal: 

• Discussed the school’s mission. 

• Conveyed high expectations for the student’s improvement. 

• Reviewed the student’s grade 7 English language arts scores and projected grade 8 
scores. 

• Set a goal for the student’s grade 8 English language arts assessment. 

• Expressed appreciation, support, and encouragement. 

Control group students engaged in similar conversations with the principal, but after the 
2009 grade 8 test. 

Findings suggested that achievement was related to group membership (r = .33). On 
average, students in the experimental group achieved higher scores than their predicted 
achievement levels compared with control group students and their predicted achievement 
levels. 

Other studies found mixed evidence for the relationships between principal characteristics and 
student achievement 

This review includes 51 other studies that meet all the requirements for inclusion (see 
appendix D for included studies and study characteristics). Of those 51 studies, 38 quanti- 
tative and 2 mixed method studies provided mixed correlations between various principal 
characteristics and student achievement. However, correlations do not indicate causality. 
Eleven qualitative studies supported the findings of the other studies. 

The study findings are organized into three categories of principal characteristics: precur- 
sors, behaviors, and leadership styles (the studies conducted in each category are shown 
in appendix E). The results for each category are summarized here with detailed findings 
below. 

Precursors. In general, evidence from studies examining the relationships between prin- 
cipal precursors (such as principals’ experience and educational attainment) and student 
achievement was positive. However, the evidence from studies on principal preparation 
programs, also a precursor, provided mixed results. 

Behaviors. Evidence from studies examining the relationships between principal behav- 
iors (such as instructional management, internal relations, organizational management, 
administrative duties, and external relations) and student achievement was mixed. Results 
suggested positive relationships between student achievement and principals’ instructional 
management, internal relations, and organization management. However, only qualita- 
tive findings suggested a relationship between principals’ external relations and student 
achievement. 

Leadership styles. Several studies examined the relationships between student achieve- 
ment and principals’ leadership styles such as distributive, collaborative, collective, colle- 
gial, learning centered, and transformational. Findings were mixed. 


In one 
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study students 
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in individual 
achievement-based 
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English language 
arts test achieved 
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Precursors: Conditions that shape leadership behaviors 


Leadership precursors are described as experience, knowledge, personal traits, and values 
and beliefs (Murphy et al, 2006). This section presents findings on precursors 5 that shape 
leadership behaviors, broadly categorized as: 

• Principals’ experience. 

• Principal preparation programs. 

• Principals’ personal traits (educational attainment). 

Principals’ experience 

Seven studies addressed the relationship between principals’ experience and student 
achievement (Braun, Gable, & Kite, 2011; Clark, Martorell, & Rockoff, 2009; Grissom 
& Loeb, 2011; Jacobson, Brooks, Giles, Johnson, & Ylimaki, 2007; Knoeppel & Rinehart, 
2008; Ruff & Shoho, 2005; Vanderhaar, Munoz, & Rodosky, 2006). 6 

Principals’ experience was defined in various ways: 

• Years of experience in education. 

• Teaching experience prior to becoming a principal. 

• Years of experience as an assistant principal at current school prior to becoming a 
principal at the school. 

• Years of experience as a principal. 

Study results in these subcategories were mixed. The exception was years of experience in 
education, where the results showed no relationship. 

Years of experience in education. Knoeppel and Rinehart (2008) defined experience as 
the number of years the principal spent in the field of education (roles were not specified). 
They found that experience in education, when analyzed with other variables, was not 
associated with student achievement. 

Teaching experience prior to becoming a principal. The effect of a principal’s years of 
experience as a teacher on improving student achievement was mixed. Clark et al. (2009) 
found that the number of years of teaching experience was not associated with student 
achievement. By contrast, Vanderhaar et al., (2006) found that achievement was higher 
among students whose principals were in the middle of their careers. Students whose prim 
cipal had 9-17 years of teaching experience had higher achievement than students whose 
principal had less than 9 years (r = .13) or more than 17 years of experience (r = .35). This 
finding suggests a relationship between a specific range of teaching experience (9-17 years) 
and student achievement, but why that specific range would matter was not examined. 

Years of experience as an assistant principal at current school prior to becoming a prim 
cipal at the school. One study examined years of experience as an assistant principal at the 
principal’s current school. This narrow definition of experience was significantly related to 
student achievement for inexperienced principals but was not significant for experienced 
principals, suggesting that over time the importance of having been an assistant principal 
at the same school diminishes (Clark et al., 2009). 


The effect of a 
principal’s years 
of experience 
as a teacher on 
improving student 
achievement 
was mixed 
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Years of experience as a principal. The six studies that focused on years of experience as 
a principal showed mixed results. While three quantitative studies suggested no significant 
relationship between length of experience as a principal and student achievement (Jacob- 
son et al, 2007; Knoeppel & Rinehart, 2008; Vanderhaar et al, 2006), one study showed 
that experience as a principal was the only significant demographic predictor of student 
academic growth (Grissom & Loeb, 201 1). 7 Two qualitative studies suggested a positive 
association (Braun et al., 2011; Ruff & Shoho, 2005). 

Principal preparation programs 

Principal preparation is an important factor in perceptions of school leadership effective- 
ness. Eight studies addressed the relationships between aspects of principal preparation pro- 
grams and student achievement (Braun et al., 2011; Clark et al., 2009; Corcoran, Schwartz, 
& Weinstein, 2012; Donmoyer, Yennire-Donmoyer, & Galloway, 2012; Kaplan, Owings, & 
Nunnery, 2005; Knoeppel & Rinehart, 2008; Owings, Kaplan, & Nunnery, 2005; Vander- 
haar et al., 2006). Three categories of principal preparation programs were studied: 

• Types of preparation programs. 

• University- and district-based programs. 

• Content of preparation programs. 

Types of preparation programs. The two studies that examined the possible effect of a 
principal being prepared through a specific type of program on student achievement 
yielded mixed results. One found that students at schools where the principal was a Cahn 
Fellow (a professional development program for principals with at least four years of expe- 
rience in high-performing schools) had marginally significantly higher English language 
arts (r < .01) and math (r = .01) scores (Clark et al., 2009). In the same study, research- 
ers found no relationship between principals participating in New York City’s Aspiring 
Principals Program (a leadership development program to prepare leaders for instruction- 
al improvement efforts in the city’s highest needs schools) and improvement in student 
achievement. Further, it found that education credentials (for example, selectivity of the 
school where principals received their degree) did not influence student achievement. In 
another study examining the Aspiring Principals Program, the effect size for annual gain 
in overall school achievement was approximately r = .01 (negligible) for schools with a 
new Aspiring Principals Program principal compared with schools with a new nonprogram 
principal (Corcoran et al., 2012). 

University - and district-based programs. No relationship was found between the princi- 
pal preparation program attended and student achievement, even controlling for principal 
experience and school contextual factors (Vanderhaar et al, 2006). 

Content of preparation programs. Five studies examined the content of preparation 
programs. 

In one study, researchers found a positive relationship between the date principals received 
their certification and student achievement (Knoeppel & Rinehart, 2008). Principals were 
categorized by whether they received their certification prior to, during, or after imple- 
mentation of standards-based reform in their state, which corresponded to the content of 
the preparation programs. Prior to implementation, principals were not assessed for spe- 
cific leadership skills; during implementation, they were required to show proficiency in 


The two studies 
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organization-related knowledge; and after implementation, principal preparation empha- 
sized the role of principals as instructional leaders. Principal preparation was a positive 
predictor of student achievement growth (r = .10), and principals who were trained after 
implementation as organizational and instructional leaders showed qualitatively larger, 
although not significant, mean achievement growth than principals who did not receive 
training related to instructional leadership. 

Three studies found evidence to support the inclusion of Interstate School Leaders 
Licensure Consortium standards in principal preparation programs (Kaplan et al., 2005; 8 
Knoeppel & Rinehart, 2008; Owings et al., 2005). 9 

In a qualitative study, Donmoyer et al. (2012) found that the overall influence of principal 
preparation programs on student achievement was mixed. One school in the study showed 
declines, another school showed gains, and the remaining two schools showed both gains 
and declines in student test scores. There was evidence that the use of cohort design, 
group case activities, and an emphasis on collaborative leadership in principal preparation 
programs had positive impacts on principals’ job performance. 

One study found a positive relationship between the content and delivery (as a combined 
construct) of principals’ training and student achievement (r = .10; Braun et al., 2011). The 
aspects of content included: 

• Emphasizing instructional improvement. 

• Emphasizing improving school and student achievement. 

• Engaging participants in problermbased learning. 

• Aligning preparation to standards of practice. 

• Covering areas needed for success in the first year of leadership. 

• Adapting preparation to participants’ individual needs. 

The aspects of delivery included: 

• Providing an excellent mentor. 

• Using cohorts. 

• Requiring reflection of participant practice and how to improve it. 

• Conducting performance assessments of skill development and leadership 
competencies. 

• Requiring an internship that is an excellent learning experience for becoming a 
principal. 

Principals’ personal traits (educational attainment) 

Two studies addressed the relationship between principals’ educational attainment and 
student achievement (Young, Vang, & Young, 2008; Valentine & Prater, 2011) with mixed 
results. Young et al. (2008) found that a principal’s education level was not a significant 
independent predictor of student achievement. In contrast, Valentine and Prater (2011) 
found that a principal’s education level had the strongest relationship to student achieve- 
ment among principal personal traits. 10 
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Behaviors: Domains of principal action 


This section describes the literature on principal behaviors linked to improved student 
achievement. Principal behaviors are defined according to Grissom and Loeb (2011), who 
used survey data from administrators, principals, and teachers in Miami-Dade County 
Public Schools in 2008 to investigate principal efficacy, leadership tasks, and the effects 
of principal efficacy on student achievement and other factors. They divided 42 principal 
tasks into five broad domains of principal action: 

• Instructional management. The promotion, support, and improvement of classroom 
instruction and school curricula. 

• Internal relations. Building strong interpersonal relationships with students, teach- 
ers, and parents. 

• Organizational management. Overseeing the budget, resources, facilities, and envi- 
ronment of the school. 

• Administrative duties. Routine, day-to-day tasks such as completing paperwork and 
managing schedules of discipline. 

• External relations. Working with stakeholders beyond the school. 

Instructional management 

Eighteen studies addressed the relationships between instructional management and 
student achievement. The four types of instructional management behaviors (Grissom 
and Loeb, 2011) and the research associated with each are discussed below. The behaviors 
include: 

• Monitoring and providing feedback to teachers and students. 

• Having a vision for learning. 

• Providing support and professional development to teachers. 

• Using data to drive decisionmaking. 


Three studies 
found positive 
effects of 
principals 
monitoring 
and providing 
feedback, and 
four studies found 
positive effects 
between principals’ 
having a vision for 
student learning 
and improved 
student learning 


Monitoring and providing feedback to teachers and students. In addition to the exper- 
imental study conducted by Silva et al. (2011), four studies found positive effects of prin- 
cipals monitoring and providing feedback. Chappelear & Price (2012) and Fancera & 
Bliss (2011) found that principals’ monitoring of and feedback to teachers were associated 
with improved student achievement. Fancera & Bliss (2011) found a relationship between 
student achievement and the principal instructional management behaviors of supervising 
and evaluating instruction and monitoring student progress (r = .19—29). Protection of 
instructional time was also significantly related to student achievement (r = .04—33). These 
findings were consistent with Williams et al.’s (2008) findings. Ovando and Ramirez (2007) 
interviewed exemplary principals and assistant principals across elementary, middle, and 
high school grade levels in Texas to identify how they perceive their instructional leader- 
ship actions in appraising teachers. The results were consistent with the broader findings of 
the relationships between instructional management behaviors and student achievement. 

Having a vision for learning. Four studies found positive effects between principals’ having 
a vision for student learning and improved student learning. 

The first study found that promoting high standards for student learning (r = .55— 61) 
and having a rigorous curriculum (r = .42—47) were most highly correlated with English 
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language arts achievement in grades 3-5 and that performance accountability was signifi- 
cantly correlated in grade 3 (r = .37; Reardon, 2011). 

A similar study revealed a positive relationship between teachers’ ratings of principals’ 
frequency of promoting the school learning climate and student achievement in English 
language arts (r = .39) and math (r = .34; O’Donnell & White, 2005). 

Sebastian & Allensworth (2012) found a significant positive indirect effect of principal 
leadership on student achievement (r = .26) and grade point average (r = .26), through 
the principals’ influence on the learning climate, which influenced classroom instruction, 
which in turn influenced student achievement. This evidence supports the theory that 
principals’ indirect effect on student achievement may be stronger than their direct effect. 

The fourth study (Brown, Benkovitz, Muttillo, & Urban, 2011), which was qualitative, 
supported the findings of the three quantitative studies: the principal having a vision for 
learning was positively associated with student achievement. 

Providing support and professional development to teachers. Research supports a rela- 
tionship between principals providing support for professional development and student 
achievement. For example, Supovitz, Strinides, and May (2010) investigated the relation- 
ships between principal leadership, peer influence, and change in instruction, and student 
achievement. Findings suggest a negligible indirect effect ((3 = .03) of principal leadership 
on English language arts scored. 11 Qualitative studies found that successful leaders: 

• Personalize instructional practices (Sherman & Crum, 2007). 

• Manage changes in the school environment (Crum & Sherman, 2008). 

• Encourage staff involvement in professional development (Borko, Wolf, & Simone, 
2003; Crum & Sherman, 2008; Sanzo, Sherman, & Clayton, 2011). 

• Practice distributive, instructional, and transformational leadership (Masumoto & 
Brown-Welty, 2009; see box 2 for definitions of key terms). 

Using data to drive decisionmaking. Findings from three qualitative studies support using 
data to drive decisionmaking. Through interviews with principals and teachers, research- 
ers explored a range of actions principals could engage in to drive instruction and staffing 
decisions. Their research supported emphasizing data-driven decisionmaking (Cohen- 
Vogel, 2011; Crum, Sherman, & Myran, 2010; Sherman and Crum, 2007). 

Internal relations 

Nine studies addressed the relationships between internal relations and student achieve- 
ment. Eight found a positive relationship (Silva et al., 2011; Leana & Pil, 2006; Horvat, 
Curci, & Chaplin, 2010; Crum & Sherman, 2008; Egley & Jones, 2005; Louis, Dretzke, & 
Wahlstrom, 2010; Sanzo et al., 2011; Sherman & Crum, 2007). Six of the nine studies are 
highlighted below. 

Silva et al. (2011) found a direct effect of school leadership on student achievement 
through interactions with students. Egley and Jones (2005) found a positive relationship 
among teachers’ perceptions of their principal’s professionally and personally inviting 
behaviors, school climate, and the accountability rating of their school. Specifically, teach- 
ers rated principals higher on professionally inviting behaviors (for example, high levels 
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of expectations from coworkers, communicating high expectations to students) than on 
personally inviting behaviors (for example, politeness and optimism). Both scales were sig- 
nificantly correlated with student achievement (r = .16 for professionally inviting behaviors 
and .09 for personally inviting behaviors). 

Leana and Pil (2006) examined the effects of internal social capital (relations with teachers) 
and external capital (relations between the principal and external stakeholders) on organi- 
zational performance as measured by student achievement on English language arts and 
math tests in 2000/01 and 2001/02. They found that internal social capital was significantly 
related to English language arts and math scores. Effect sizes for these correlations ranged 
from r = .16 to r = .50, depending on the year, subject, and internal relationship studied. 

Other researchers identified common behaviors of successful school principals, such as 
engaging parents and the community while maintaining overall authority of the school 
(Horvat et al., 2010) and communicating and developing a rapport with teachers and stu- 
dents (Crum & Sherman, 2008). These findings are consistent with the findings of the 
quantitative studies. 

However, one study did not find a relationship between principal behaviors regarding 
internal relations (for example, district support or diversity of the building council) and 
student achievement (Gordon & Louis, 2009). 

Organizational management 

Five studies addressed the relationships between principals’ time spent on and the effec- 
tiveness of their organizational management tasks and student achievement (Bloom & 
Owens, 2013; Cohen-Vogel, 2011; Grissom & Loeb, 2011; Horng, Klasik, & Loeb, 2010; 
May, Huff, & Goldring, 2012). 

Three of the five studies found that the percentage of time principals spent on organiza- 
tional management tasks was positively associated with student achievement (Horng et al., 

2010) and principals’ effectiveness at organizational management tasks (Grissom & Loeb, 

2011) . For example, Grissom and Loeb’s (2011) study found that organizational manage- 
ment was the only significant predictor of school accountability performance (r = .15) and 
school accountability performance gains (r = .13). Organizational management was also a 
significant predictor of student achievement in English language arts (r = .01) and math 
(r = .01), though the effect size was negligible. May et al. (2012) also found a positive rela- 
tionship between the amount of time principals spent on school finance and personnel 
issues and student achievement. 

Qualitative findings from two studies provided support for principals’ engagement in orga- 
nizational management tasks, such as staffing decisions and school funds (Bloom & Owens, 
2013) and using student achievement data to make staffing decisions (Cohen-Vogel, 2011). 

Administrative duties 

None of the studies in the review provided direct evidence for relationships between 
administrative duties and student achievement. However, in examining the relationship 
between teachers’ perceptions of principal leadership and student achievement, Williams 
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(2009) found that student achievement was significantly correlated with planning and 
organization (r = .04) and school climate (r = .05). This may provide support for indirect 
relationships between administrative duties and student achievement. 

External relations 

Three studies examined the relationships between external relations and student achieve- 
ment with mixed results (Gordon & Louis, 2009; Horvat et al., 2010; Leana & Pil, 2006). 
Gordon and Louis (2009) did not find any significant relationship between site council diver- 
sity, district support, or the principal’s openness to the community and student achievement. 
By contrast, Masumoto and Brown-Welty’s (2009) qualitative cross case study of the practices 
of leaders of successful, high-poverty, rural schools showed that school-community links that 
address the school’s mission enhance student achievement. This finding is consistent with 
Horvat et al.’s (2010) finding that engaging parents and the community enabled principals to 
leverage their resources and increase the leadership power to improve student achievement. 

Leadership styles 


This section describes principals’ leadership styles and their potential relationships to 
student achievement. Researchers have linked principals’ leadership styles to improved 
student achievement. While the studies examined in this review largely fit within Grissom 
and Loeb’s (2011) instructional management domain, 13 studies explored different aspects 
of leadership styles (Griffith, 2004; Hallinger & Heck, 2010a, 2010b, 2011; Heck & Hal- 
linger, 2009, 2010a, 2010b; Heck & Moriyama 2010; Leithwood & Jantzi, 2008; Leithwood 
& Mascall, 2008; Marks & Printy, 2003; Tschannen-Moran, Parish, & DiPaola, 2006; Twigg, 
2008). Within these studies, there was considerable overlap in leadership styles as well as 
mixed evidence of effects on student learning. (See box 2 for definitions of leadership styles.) 

Distributive, collaborative, and learning-directed leadership 

In 2009 Hallinger, Heck, and others began publishing a series of studies modeling the 
effects of learning-directed leadership broadly, 12 and distributed or collaborative leadership 
specifically, on student achievement. The studies focused on leadership 13 related to: 

• Making collaborative decisions focusing on education improvement. 

• Emphasizing school governance that empowers staff and students and encourages 
commitment. 

• Implementing broad participation and shared accountability for student learning. 

• Emphasizing efforts to evaluate school academic development based on student 
achievement in the context of educational effectiveness research. 

Each study used a random sample of data from a larger dataset of public elementary schools 
measuring student achievement beginning in grade 3 for three years. Yearly survey data on 
teacher perceptions of school leadership and academic capacity were also included. 

Collectively, the studies found no evidence of a direct effect of collaborative or distribut- 
ed leadership on student achievement but consistently found significant indirect effects. 
Specifically, they found that changes in collaborative or distributed leadership had a sig- 
nificant effect on changes in school academic capacity. That change in academic capacity 
in turn had a significant effect on growth in student achievement in English language 
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arts (Hallinger & Heck, 2010a, 2010b, Heck & Hallinger, 2010b) and math (Hallinger & 
Heck, 2010a; Heck & Hallinger, 2009, 2010a, 2010b). Heck and Moriyama (2010) produced 
similar findings showing a significant indirect effect of distributed leadership on added year 
effects in and English language arts and math via instructional practice. 

Additionally, the researchers found that the aforementioned effects were reciprocal and 
reinforcing. That is, the change in achievement directly affected change in school aca- 
demic capacity which affected change in leadership in a feedback loop (Hallinger & Heck, 
2010a; Heck & Hallinger, 2010a). 

Additional research on leadership styles 

Other researchers (Leithwood & Jantzi, 2008; Leithwood & Mascall, 2008; Marks & Printy, 
2003) used leadership styles to investigate the relationships between school leadership and 
student achievement. Leithwood and Jantzi (2008) examined the relationships between lead- 
ership self-efficacy and leadership collective efficacy and student achievement. Leadership 
self-efficacy measures school leaders’ beliefs about their own ability to drive school improve- 
ment; leadership collective efficacy measures school leaders’ beliefs about the efficacy of a dis- 
trict leadership’s role in school improvement. Combined, leadership efficacy was significantly 
correlated with the percentage of students reaching proficiency in English language arts and 
math but not with mean achievement gain in 2003-05. Significant positive indirect effects 
of school leader behavior ((2 = 0.24) and leadership collective efficacy ((3 = 0.32) were found 
on student achievement. 14 Leadership self-efficacy had significant negative indirect effects 
on student achievement; however, the effects were very small ((3 = -0.01). Similar research 
found that collective leadership, defined as “combined effects” of various sources of school 
leadership, was associated with student achievement (r = .34; Leithwood & Mascall, 2008). 

Other research studies emphasized transformative and collegial school leadership. For 
example, Tschannen-Moran et al. (2006) used an organizational climate framework to 
investigate the relationships between teachers’ perceptions of school climate variables 
(that is, collegial leadership, teacher professionalism, academic press, and community 
engagement) and student achievement. Collegial leadership, which is characterized as sup- 
portive behavior of teachers and collegiality between school leaders and teachers, was not 
significantly correlated with student achievement. 

Other researchers found evidence suggesting that a transformational leadership style pos- 
itively affects student achievement (Marks & Printy, 2003; Griffith, 2004; Twigg, 2008). 
Marks and Printy (2003) measured the effects of different leadership styles on student 
achievement and instructional quality. Schools with a higher level of integrated leader- 
ship (that is, transformational and shared leadership) had higher academic achievement 
((3 = 0.56) than schools with a lower level of integrated leadership. 15 In examining the 
effects of transformational leadership behaviors on school staff turnover and student 
achievement, Griffith (2004) found that transformative leadership indirectly, yet signifi- 
cantly, predicted student achievement. In exploring the extent to which transformative 
leadership influenced student achievement and teachers’ citizen behaviors (for example, 
attending nonrequired development functions, speaking well of the school), Twigg (2008) 
found that transformative leadership had a significant indirect effect on student achieve- 
ment through perceived organizational support, self-esteem, and citizen behaviors, with a 
total indirect effect of = 0.32. 16 
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Implications for state and local education agencies 


These findings can guide discussions about variations in principal effectiveness. Under- 
standing which principal characteristics are associated with student achievement can 
provide insight into why some principals are more effective than others. Given the number 
of principals’ roles, responsibilities, and tasks, it is beneficial to know specifically which 
factors have the most impact on student achievement. 


School leaders can also benefit from research on the outcomes of their time and effort. For 
example, Silva et al. (2001) found that having discussions with students about their per- 
formance prior to state assessments had a positive impact on their achievement. Although 
this activity may seem time consuming, it may have more effect on student achievement 
than other tasks, such as administrative duties. 

Findings from this review can also inform policies on professional development for aspir- 
ing (preservice) and practicing (inservice) school leaders. Evidence reported in this review 
suggests that the training content of principal preparation programs is influenced by state 
standards for certification at the time. Evidence also supported the inclusion of specific 
content and activities in principal preparation programs. State and districts might con- 
sider reviewing and modifying programs to include aspects of preparation programs that 
are associated with increased student achievement. Practitioners who want to apply these 
findings should review the individual studies cited to better understand their unique char- 
acteristics (see appendix D for more details on each study). 
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More research is needed on whether the benefits of specific characteristics might vary in 
different settings and contexts. Relevant questions to drive this research might include: 
are the relationships the same in large versus small, rural versus urban, or elementary 
versus high school settings? The evidence base would benefit from rigorous studies exam- 
ining precursors and behaviors for principals serving in these multiple settings. Similarly, 
more research is needed on effective characteristics of school leaders in other settings (for 
example, virtual classrooms, charter schools, and alternative schools). 


Study limitations 


Despite the large amount of research conducted on school leadership and student achieve- 
ment, only a few studies met the rigor necessary to recommend application in practice 
or policy. Only one study was a randomized controlled trial (Silva et al., 2011). That 
study stands out because experimental studies are the only type of study that can provide 
credible evidence on whether an intervention is effective. Further, it demonstrates that 
although there are many challenges to using experimental designs in research on principal 
effectiveness, it is possible to design research that can test experimentally the relationships 
between principal behaviors and student achievement. 

The search criteria included only studies that had been subject to peer review. Therefore, 
studies such as technical reports that were subject to peer review were included, but disser- 
tations and the like were not. Thus, publication bias is a limitation because unpublished 
works — which often have nonsignificant or otherwise nondesirable negative results — were 
not included. 
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Appendix A. Results of previous reviews and meta-analyses 


This appendix provides supplemental tables reporting results of previous systematic reviews 
or meta-analyses. 


Table Al. Mean correlation of five leadership dimensions with student achievement and 
nonacademic outcomes 

1 Leadership dimension 

Meaning of dimension 

Mean effect size j 

Establishing goals and expectations 

Setting, communicating, and monitoring learning goals, 
standards, and expectations and involving staff and others in 
the process so that there is clarity and consensus about goals. 

r= .40 

Resourcing strategically 

Aligning resource selection and allocation to priority teaching 
goals and providing appropriate expertise through staff 
recruitment. 

r= .30 


Planning, coordinating, and evaluating 
teaching and the curriculum 

Being directly involved in the support and evaluation of teaching 
through regular classroom visits and the provision of formative 
and summative feedback to teachers. 

r= .40 


Having direct oversight of curriculum through schoolwide 
coordination across classes and year levels and alignment to 
school goals. 


Promoting and participating in teacher 
learning and development 

Not only promoting but also directly participating with teachers 
in formal or informal professional learning. 

r= .69 

Ensuring an orderly and supportive 
environment 

Protecting time for teaching and learning by reducing external 
pressures and interruptions and establishing an orderly and 
supportive environment both inside and outside classrooms. 

r= .26 

Source: Robinson, Hohepa, & Lloyd, 2009. 
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Table A2. Mean correlation of Balanced Leadership Framework’s 21 principal responsibilities with 

student achievement, matched with Cotton’s 25 principal practices 



Balanced Leadership Framework s 21 responsibilities 
with associated Cotton s (2003) 25 practices 

Balanced Leadership Framework responsibility 
description: The extent to which the principal... 

P 

e 

riean 

ffect 

size 

Affirmation 

Recognizes and celebrates accomplishments and 

r 

= .19 

Rituals, ceremonies, and other symbolic actions 

acknowledges failures 



Recognition of student and staff achievement 




Change agent 

Is willing to challenge and actively challenges the status 

r 

= .25 

Support of risk taking 

quo 



Contingent rewards 

Recognizes and rewards individual accomplishments 

r 

= .24 

Rituals, ceremonies, and other symbolic actions 




Recognition of student and staff achievement 




Communication 

Establishes strong lines of communication with and 

r 

= .23 

Communication and interaction 

among teachers and students 



Shared leadership, decisionmaking, and staff empowerment 




Culture 

Fosters shared beliefs and a sense of community and 

r 

= .25 

Positive and supportive school climate 

cooperation 



Collaboration 




Discipline 

Protects teachers from issues and influences that 

r 

= .27 

Protecting instructional time 

would detract from their teaching time or focus 



Flexibility 

Adapts his or her leadership behavior to the needs of 

r 

= .28 

Support of teacher autonomy 

the current situation and is comfortable with dissent 



Focus 

Establishes clear goals and keeps those goals in the 

r 

= .24 

Vision and goals focused on high levels of student learning 

forefront of the school’s attention 



High expectations for student learning 




Ongoing pursuits of high levels of student learning 




Norm of continuous improvement 




Monitoring student progress and sharing findings 




Ideals/beliefs 

Communicates and operates from strong ideals and 

r 

= .22 

Self-confidence, responsibility, and perseverance 

beliefs about schooling 



Input 

Involves teachers in the design and implementation of 

r 

= .25 

Visibility and accessibility 

important decisions and policies 



Shared leadership, decisionmaking, and staff empowerment 




Intellectual stimulation 

Ensures faculty and staff are aware of the most current 

r 

= .24 

Norm of continuous improvement 
Discussion of instructional issues 

theories and practices and makes the discussion of 
these a regular aspect of the school's culture 



Involvement in curriculum, instruction, and assessment 

Is directly involved in the design and implementation of 

r 

= .20 

Instructional leadership 

curriculum, instruction, and assessment practices 



Classroom observation and feedback to teachers 




Role modeling 




Knowledge of curriculum, instruction, and assessment 

Is knowledgeable about current curriculum, instruction, 

r 

= .25 

Instructional leadership 

and assessment practices 



Role modeling 





(continued) 
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Table A2. Mean correlation of Balanced Leadership Framework’s 21 principal responsibilities with 
student achievement, matched with Cotton’s 25 principal practices (continued) 

Balanced Leadership Framework s 21 responsibilities 
with associated Cotton s (2003) 25 practices 

Balanced Leadership Framework responsibility 
description: The extent to which the principal... 

Mean 

effect 

size 

Monitoring/evaluating 

Classroom observation and feedback to teachers 
Monitoring student progress and sharing findings 
Use of student progress data for program improvement 

Monitors the effectiveness of school practices and their 
impact on student learning 

II 

ro 

Optimizer 

Vision and goals focused on high levels of student learning 
Self-confidence, responsibility, and perseverance 
Ongoing pursuits of high levels of student learning 

Inspires and leads new and challenging innovations 

r = .20 

Order 

Safe and orderly school environment 

Establishes a set of standard operating procedures and 
routines 

r= .25 

Outreach 

Parent and community outreach and involvement 

Is an advocate and spokesperson for the school to all 
stakeholders 

r= .27 

Relationships 

Communication and interaction 
Emotional and interpersonal support 

Demonstrates an awareness of the personal aspects of 
teachers and staff 

r = .18 

Resources 

Professional development opportunities and resources 

Provides teachers with materials and professional 
development necessary for the successful execution of 
their jobs 

r= .25 

Situational awareness a 

Is aware of the details and undercurrents in the running 
of the school and uses this information to address 
current and potential problems 

r = .33 

Visibility 

Visibility and accessibility 
Emotional and interpersonal support 

Has quality contact and interactions with teachers and 
students 

r= .20 

a. Marzano et al. (2005) do not identify any of Cotton’s (2003) leadership practices that align with situational awareness. 
Source: Marzano et al., 2005. 
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Table A3. Essential school leadership preparation practices and supporting literature 


Practice Supporting research 


Structure 

Partnerships between 
universities and 
districts 

Darling-Hammond, LaPoint, Meyerson, Orr, & Cohen, 2007; Jackson & Kelley, 2002; Milstein & Krueger, 
1997; Murphy, 1993a, 1993b, 1999; Orr, 2006; Southern Regional Educational Board, 2006; U.S. 
Department of Education, 2004 

Program developers' 
commitment 

Darling-Hammond et al., 2007; U.S. Department of Education, 2004 

Rigorous entrance 
requirements for strong 
and diverse candidates 

Bredeson, 1996; Darling-Hammond et al., 2007; Hart & Pounder, 1999; Leithwood & Jantzi, 1996; 
Milstein & Krueger, 1997; Murphy, 1993a, 1993b; Murphy et al., 2006; Orr, 2006; Southern Regional 
Educational Board, 2006; U.S. Department of Education, 2004 

Financial support, 
release time for 
participants 

Darling-Hammond et al., 2007; Leithwood & Jantzi, 1996; Milstein & Krueger, 1997; Southern Regional 
Educational Board, 2006 

Supportive district and 
state infrastructure 

Darling-Hammond et al., 2007; Orr, 2006; Southern Regional Educational Board, 2006 

Program monitoring for 

improvement 

Content 

Lauder, 2000; Leithwood & Jantzi, 1996; Milstein & Krueger, 1997; Murphy, 1999; Orr, 2006; Southern 
Regional Educational Board, 2006; U.S. Department of Education, 2004 

Standards-based 

content 

Darling-Hammond et al., 2007; Jackson & Kelley, 2002; Lauder, 2000; Orr, 2006; Southern Regional 
Educational Board, 2006; U.S. Department of Education, 2004 

Coherent and relevant 
curriculum 

Darling-Hammond et al., 2007; Jackson & Kelley, 2002; Milstein & Krueger, 1997; Orr, 2006; Southern 
Regional Educational Board, 2006; U.S. Department of Education, 2004 

Individualized content 

Jackson & Kelley, 2002; Lauder, 2000; Leithwood & Jantzi, 1996; Murphy, 1993a 

Focus on shared 
instructional leadership 

Elmore, 1999; Jackson & Kelley, 2002; LaPoint, Meyerson, & Darling-Hammond, 2005; Leithwood & 
Jantzi, 1996; McCarthy, 1999; Murphy, 1993b, 1999; Orr, 2006; Southern Regional Educational Board, 
2006 

Focus on school reform 
or social justice 
Delivery 

Jackson & Kelley, 2002; LaPoint et al., 2005; Leithwood & Jantzi, 1996; McCarthy, 1999; Murphy, 1993b, 
1999; Orr, 2006; Southern Regional Educational Board, 2006 

High-quality internship 

Bredeson, 1996; Darling-Hammond et al., 2007; Hart & Pounder, 1999; Jackson & Kelley, 2002; Lauder, 
2000; Leithwood & Jantzi, 1996; Leithwood, Louis, Anderson, & Wahlstrom, 2004; Milstein & Krueger, 
1997; Murphy, 1993b, 1999; Murphy et al., 2006; Southern Regional Educational Board, 2006; U.S. 
Department of Education, 2004 

Problem-based learning 

Darling-Hammond et al., 2007; Hart & Pounder, 1999; Jackson & Kelley, 2002; Lauder, 2000; Leithwood & 
Jantzi, 1996; Leithwood et al., 2004; McCarthy, 1999; Murphy, 1993a, 1993b, 1999; Orr, 2006; Southern 
Regional Educational Board, 2006 

Mentoring or coaching 

Darling-Hammond et al., 2007; Jackson & Kelley, 2002; Lauder, 2000; Leithwood & Jantzi, 1996; Milstein 
& Krueger, 1997; Murphy, 1993a; Southern Regional Educational Board, 2006 

Cohort structure 

Darling-Hammond et al., 2007; Hart & Pounder, 1999; Jackson & Kelley, 2002; Leithwood & Jantzi, 1996; 
McCarthy, 1999; Milstein & Krueger, 1997; Murphy, 1993b; U.S. Department of Education, 2004 

Habit of reflection 

Davis, Darling-Hammond, LaPoint, & Meyerson, 2005; LaPoint et al., 2005; Lauder, 2000; Leithwood & 
Jantzi, 1996; Milstein & Krueger, 1997; Murphy, 1993b; Southern Regional Educational Board, 2006 

Performance 

assessments 

Hart & Pounder, 1999; Jackson & Kelley, 2002; Lauder, 2000; Leithwood & Jantzi, 1996; Murphy, 1993b; 
Orr, 2006; Southern Regional Educational Board, 2006; U.S. Department of Education, 2004 


Source: Braun et al., 2011 
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Appendix B. Theoretical frameworks 


The figures in this appendix depict the theoretical frameworks that guided this systematic 
review. 


Figure Bl. Theoretical research framework linking leadership to learning 



Source: Leithwood et at, 2004. 


Figure B2. Direct, indirect, and reciprocal effects models of principal effects 


Direct effects model 


Principal leadership 


Student achievement 


Direct effects with antecedent effects 



Indirect effects model 



Indirect effects with antecedent effects 



Reciprocal effects model 



Source: Adapted from Hallinger and Heck (1998). 
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Figure B3. Learning-centered framework explaining the precursors to leadership 
behaviors with a potential influence pathway to student outcomes 


Influence 


Precursors Behaviors pathway Outcomes 



Context within the state, district, and school 

(for example, accountability, external conditions, standards, 
curriculum, instruction, culture, school type, school level, nature, 
student composition, staff composition) 


Source: Murphy et al., 2006. 


Figure B4. Leadership knowledge, skills, and dispositions that support effective 
leadership 


Dimensions from 
direct evidence 


Establishing goals 
and expectations 


Resourcing strategically 


Planning, coordinating, 
and evaluating teaching 
and the curriculum 


Promoting and 
participating in teacher 
learning and development 


Ensuring an orderly and 
supportive environment 


Leadership knowledge, 
skills, and dispositions 


Ensuring administrative 
decisions are informed 
by knowledge about 
effective pedagogy 


Analyzing and solving 
complex problems 


Building relational trust 


Engaging in 
open-to-learning 
conversations 


Dimensions from 
indirect evidence 


Setting educational goals 


Obtaining and allocating 
resources aligned to 
pedagogical goals 


Creating educationally 
powerful connections 


Creating a community 
that learns how to 
improve student success 


Engaging in constructive 
problem talk 


Selecting, developing, 
and using smart tools 


Source: Robinson etal., 2009. 
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Appendix C. Systematic review protocol 


This appendix describes the methodology used to complete this systematic review. 

Eligibility criteria 

To be included in the review, a study had to meet several relevance criteria: 

Language relevance. The literature had to be written in English. 

Outcome relevance. The study had to: 

• Focus on quantifiable student achievement; student achievement could be aggre- 
gated to the school level. 

• Include an observable principal characteristic that was quantified, operationally 
defined, or sufficiently described so that it would be replicable. 

Sample relevance. The sample had to include both students and principals in the K-12 
system. Studies that focused only on principal characteristics without an analysis linking 
principal characteristics to student achievement were excluded. Studies that focused exclu- 
sively on specific subpopulations (for example, only female principals, English learner stu- 
dents, or students in the juvenile justice system) or exclusively on unique school settings 
(for example, juvenile justice system schools, special education-only schools, alternative 
schools, or charter schools) were excluded. 

Study design relevance. Study design and focus were limited to empirical studies using 
quantitative or qualitative methods that illuminate relations between principal char- 
acteristics and student achievement. While experimental design research is the “gold 
standard” in education research, the questions addressed by this literature review were cor- 
relational in nature. For that reason, it was not expected that many, if any, experimental 
studies would be included. However, the review team found one experimental study. The 
review team also found 51 correlational quantitative research and descriptive qualitative 
research studies that identified relationships between principal characteristics and student 
achievement. 

Timeframe relevance. The most recent comprehensive meta-analysis on principal effec- 
tiveness, conducted by Waters et al. (2003) included research from the 1970s through 2001. 
Thus, the current systematic review included research only from 2001 to the present. 

Topic relevance. The study had to include at least one principal characteristic related to 
student achievement in the K-12 system. The study was required to focus on the direct 
effect of the principal characteristic, not the potential indirect effect. 

Literature search strategy 

The literature that was reviewed came from two sources. First, the Florida Department of 
Education provided a list (n = 82) of literature used to develop the Florida Principal head- 
ership Standards. Second, the review team conducted an independent literature search, 
including database searches, ancestral searches, and hand searches. 
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Keyword search. The following Boolean parameters were used in computer database 
searches: (“Education* leader*” OR “principal”) AND (“student achievement” OR “student 
outcomes”), limited to publication years between 2001 and 2012. Applying the Boolean 
search in full-text yielded far too many irrelevant citations, so the search was applied only 
to the fields of title, abstract, and keyword. Where possible, results were limited to peer- 
reviewed citations. The following electronic databases were used in the database search: 

• Campbell Collection. 

• Education Index Retrospective and Education Full Text. 

• ERIC. 

• ISI Web of Knowledge. 

• JSTOR. 

• PsychINFO. 

Ancestral search. The following meta-analyses and literature reviews were used to identify 
previously reviewed literature and literature that has subsequently cited the meta-analyses 
and literature reviews: 

• Cotton, K. (2003). Principals and student achievement: What the research says. Alex- 
andria, VA: Association for Supervision and Curriculum Development. 

• Davis, S. H., Darling-Hammond, L., La Point, M., & Meyerson, D. (2005). School 
leadership study: Developing successful principals. Stanford, CA: Stanford Universi- 
ty, Stanford Educational Leadership Institute. 

• Marzano, R. J., Waters, T., & McNulty, B. A. (2005). School leadership that works: 
From research to results. Alexandria, VA: Association for Supervision and Curric- 
ulum Development. 

• Mills, L. B., McDowelle, J. O., & Rouse, W. A. (2011). A meta-analysis of research 
on the mediated effects of principal leadership on student achievement. In E. H. 
Reames & M. Barakat (Eds.), Southern Regional Council on Educational Admim 
istration 2011 yearbook: Leading in the decade of challenges and opportunities (pp. 
23-30). Auburn, AL: Auburn University. 

• Robinson, V. M. J., Hohepa, M., & Lloyd, C. A. (2009). School leadership and 
student outcomes: Identifying vuhat works and why. Wellington, New Zealand: Min- 
istry of Education. 

• Robinson, V. M. J., Lloyd, C. A., & Rowe, K. J. (2008). The impact of leadership 
on student outcomes: An analysis of the differential effects of leadership types. 
Educational Administration Quarterly, 44(5), 635-674. 

• Waters, T., Marzano, R. J., & McNulty, B. (2003). Balanced leadership: What 30 
years of research tells us about the effect of leadership on student achievement. A 
working paper. Denver, CO: Mid-continent Research for Education and Learning. 

• Witziers, B., Bosker, R. J., & Kruger, M. L. (2003). Educational leadership and 
student achievement: The elusive search for an association. Educational Adminis- 
tration Quarterly, 39(3), 398-425. 

Hand search. The following list of educational leadership and policy journals were hand 
searched for additional literature: 

• American Educational Research Journal. 

• American Journal of Education. 

• Educational Administration Quarterly. 

• Educational Evaluation and Policy Analysis. 

• Educational Policy. 
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• International Journal of Educational Management. 

• Journal of Educational Administration. 

• Journal of Education Finance. 

• Journal of Education Policy. 

• Journal of School Leadership. 

• School Effectiveness and School Improvement. 

Screening strategy 

References were collected and stored in EndNote X4 software during the search process. 
Citations and full-text documents were imported into EPPI-Reviewer4 software for screen- 
ing and coding. Screening was led by the second author. All sources were double screened 
on title and abstract, and, if necessary on full text. Discrepancies in screening and coding 
were resolved through discussion. 

Coding strategy 

Sources selected for inclusion were double coded for study characteristics (that is, method- 
ology, settings, participants, and measures), study focus, principal characteristics, student 
achievement measures, and results. Coding was done in EPPI-Reviewer4 software. Interrat- 
er reliability was calculated on 20 percent of studies; Cohen’s kappa ranged from .93 to 1.0. 

Review strategy 

In advance of the search and review efforts, the review team anticipated needing to group 
studies according to common themes to make this report useful to policymakers and prac- 
titioners. In part, the more studies that pertain to a characteristic, the more confidence the 
review team could have about the results. While a guiding framework was used, the review 
team did not limit the groups according to older research categories; the groups were not 
predefined but were developed during the coding of the literature. 

Effect size formulas 

Although the most common effect size is the standardized mean difference, or Cohen’s d, 
in this review the relationships between principal characteristics and student achievement 
are of interest, so the preferred effect size is r, which estimates the magnitude of the rela- 
tionship between two variables rather than the magnitude of group differences on a single 
variable. Throughout this review, r is provided when applicable. Basic formulas for calcu- 
lating r are presented below. A variety of algebraically equivalent formulas are available to 
calculate effect sizes using other study-reported statistics; see Lipsey and Wilson (2001) for 
a full list of effect size formulas. 


Correlational effect size • The magnitude of the relationship between variables can be 
calculated as a correlational effect size, r, using the covariances between the two variables 
and their respective standard deviations. Equation Cl displays the basic Pearson product- 
moment correlation effect size using covariances and standard deviations: 


r 


xy 


cr cr 


(Cl) 
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Fisher’s z-trans formation. Some studies report the correlational effect size as a Fisher’s z- 
Fisher’s z can be transformed back to an r effect size using equation C2: 


e zz - 1 
e 2Z + 1 


(C2) 


Standardized mean difference effect size. For calculating standardized mean difference 
effect size (Cohen’s d ) group means, standard deviations, and n’s are the most commonly 
reported and used statistics. Equation C3 displays the basic formula for calculating the d 
effect size using group means, standard deviations, and n’s: 


d = 



spooled 


S 


pooled 



l)s 2 + (n 2 - l)s 2 


n 2 - 2 


(C3) 


Converting Cohen’s d to r. Some studies report an effect size as a Cohen’s d. Cohen’s d can 
be transformed back to an r effect size using equation C4: 


(n : - l)s z + (n 2 - 1 )s 


pooled 


n 2 ~ 2 


(C4) 
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Appendix D. Summary of the search and 
screening process and included studies 


This appendix provides additional information on the literature search and screening 
process (figure Dl) and details of the studies included in this systematic review (table Dl). 


Figure Dl. Flow chart of literature search and screening process 
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Table Dl. Characteristics of included studies 

Study 

Methodology 

Type of analysis 

Length of 
study (years) 

Location 

Setting 

Grade level 

Participants 

Principal data source 

Student data source 

Bloom & 
Owens, 2013 

Quantitative 

Correlational: 

Regression 

Up to 1 

Nationwide 

Urban 

High school 

• Principal n ~ 14,000 

• Self-report 

School-level, state 
test 

Borko, Wolf, & 
Simone, 2003 

Qualitative 

Qualitative: 
Case study 

Up to 1 

Washington 

State 

Mixed 

Elementary 

school 

■ School n = 2 

■ Principal n = 2 
• Teacher n = 2 

• Self-report 
Teacher or staff 
report 

• Student report 
Direct observation 

School-level, state 
test: Essential 
Academic Learning 
Requirements, 
Washington 
Assessment of 
Student Learning 

Braun, Gable, 
& Kite, 2011 

Quantitative 

Correlational: 

Regression 

Up to 1 

Rhode Island 

Mixed 

K-8 

■ Principal n = 88 

• Self-report 

School-level, state 
test: New England 
Common Assessment 
Program, English 
language arts 

Brown, 
Benkovitz, 
Muttillo, & 
Urban, 2011 

Qualitative 

Case study 

Up to 1 

Southeast 

Region 


Elementary 

school 

• District n = 1 

■ School n = 24 

• Principal n = 16 

• Assistant principal 
n = 24 

• Teacher n = 48 

• Parent n = 24 

• Self-report 
Teacher or staff 
report 

■ Parent report 

School-level, state 
test: English language 
arts and math 

Chappelear & 
Price, 2012 

Quantitative 

Correlational: 
Analysis of 
variance 

Up to 1 

Ohio/Region 

12 


High school 

• District or region 
n = 1 

■ School n = 44 

• Teacher n = 213 

Teacher or staff 
report 

School-level, 
state test: Ohio 
Achievement 
Assessment, Ohio 
Graduation Test 

Clark, 

Martorell, & 
Rockoff, 2009 

Quantitative 

Correlational: 

Regression 

Up to 6 

New York City 


Grades 3-8 

• Principal n = more 
than 1,000 

District or state 
data 

School-level, state 
test: English language 
arts and math 


(continued) 




D-3 


Table Dl. Characteristics of included studies (continued) 

Study 

Methodology 

Type of analysis 

Length of 
study (years) 

Location 

Setting 

Grade level 

Participants 

Principal data source 

Student data source 

Cohen-Vogel, 

2011 

Qualitative 

Case study 

Up to 1 

Florida 

Mixed 

Elementary 

school 

• District n = 5 

• School n = 5 

• Superintendent 
or deputy 

superintendent n = 8 

• Human resource 
director n = 5 

• Assistant principal 
or principal n = 15 

• Teacher n = 27 

• Parent n = 6 

• Self-report 

• Other administrator 
report 

Teacher or staff 
report 

Parent report 

School-level, 
state test: Florida 
accountability model 

Corcoran, 
Schwartz, & 
Weinstein, 
2012 

Quantitative 


3 

New York City 

Urban 

Grades 3-8 

• District n = 1 

■ School n = 440 

• Principal n = 440 

District or state 
data 

School-level, state 
test: New York State 
exams in English 
language arts and 
math 

Crum & 

Sherman, 

2008 

Qualitative 

Case study 

Up to 1 

Virginia 


High school 

• Principal n = 12 

• Self-report 

School-level, state 
test: Virginia’s 
Standards of Learning 
accountability system 
School-level, other: 
School accreditation, 
adequate yearly 
progress recognized 

Crum, 

Sherman, & 
Myran, 2010 

Qualitative 

Case study 

Up to 1 

Virginia 

Mixed 

Elementary 

school 

■ Principal n = 12 

• Self-report 

School-level, state 
test: Virginia’s 
Standards of Learning 
accountability system 
School-level, other: 
School accreditation, 
adequate yearly 
progress recognized 

Donmoyer, 
Yennire- 
Donmoyer, & 
Galloway, 2012 

Mixed method 

Correlational: 
Correlations only 
Qualitative: 

Case study 

Up to 2 


Urban 

Elementary 

school 

District n = 1 
■ School n = 4 

• Principal n = 4 

• Teacher n = 167 

• Staff member n = 72 

• Self-report 

• Other administrator 
report 

Teacher or staff 
report 

• Other: School 
and program 
documents 

School-level, state 
test: English language 
arts, math, and 
science 

(continued) 
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Table Dl. 

Characteristics of included studies (continued) 






Study 

Methodology 

Type of analysis 

Length of 
study (years) 

Location 

Setting 

Grade level 

Participants 

Principal data source 

Student data source 

Egley & Jones, Quantitative 
2005 

Correlational: 
Analysis of 
variance 

Up to 1 

Florida 

Mixed 

Elementary 

school 

• District n = 30 

• School n = 645 

• Teacher n = 708 

Teacher or staff 
report 

School-level, 
state test: Florida 
accountability model 

Fancera & 
Bliss, 2011 

Quantitative 

Correlational: 
Path analysis 

Up to 1 

New Jersey 

Mixed 

High school 

• County n = 18 

• School n = 53 

Teacher or staff 
report 

School-level, state 
test: New Jersey High 


Teacher n = 1,083 


School Proficiency 
Assessment on 
English language arts 
and math 
School-level, other 
standardized test: 
SAT critical reading, 
writing, and math 
School-level, other: 
Student participation 
in advanced 
placement courses 


Gordon & 

Quantitative 

Correlational: 3 

— 

Mixed 

Elementary, 

• State n = 9 

• Self-report 

School-level, state 

Louis, 2009 


Latent variables 



middle, and high 

• District n = 45 

• Other administrator 

test: English language 



and regression 



school 

• School n = 180 

report 

arts and math 







• Principal n = 157 

Teacher or staff 








• Assistant principal 

report 








n = 103 

• Other: 








• Teacher n = 4,491 

stakeholders at 
all levels including 
state, district, 
community 
including parents, 
business 
members, and 
community group 
members, and 
school. 



Griffith, 2004 

Quantitative 

Correlational: 

Up to 1 

— 

Suburban Elementary 

• School n = 117 

Teacher or staff 

School-level, state 



Structural 



school 

• School-based staff 

report 

test: English language 



equation 




n = 3,291 

Parent report 

arts and math 



modeling, 
hierarchical 
linear model 




• Student n = 25,087 

• Student report 



(continued) 
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Table Dl. Characteristics of included studies (continued) 


Study 

Methodology 

Type of analysis 

Length of 
study (years) 

Location 

Setting 

Grade level 

Participants 

Principal data source 

Student data source 

Grissom & 
Loeb, 2011 

Quantitative 

Correlational: 

Regression 

Up to 1 

Miami-Dade 
County Public 
Schools, 
Florida 

Urban 

Elementary, 
middle, and high 
school 


State n = 1 
District n = 1 
Principal n = 314 
Assistant principal 
n = 585 

Teacher n = 15,842 

• Self-report 

• Other administrator 
report 

Teacher or staff 
report 

Parent report 

School-level, 
state test: Florida 
accountability model 

Hallinger & 
Heck, 2010a 

Quantitative 

Correlational: 
Latent change 
analysis 

4 

West Region 


Elementary 

school 


State n = 1 
School n = 198 
Student 
n = 13,000+ 

Teacher or staff 
report 

Student-level, state 
test: English language 
arts and math 

Hallinger & 
Heck, 2010b 

Quantitative 

Correlational: 
Latent change 
analysis 

4 



Elementary 

school 


State n = 1 
School n = 192 
Student n = 12,480 

Teacher or staff 
report 

■ Parent report 

Student-level, state 
test: SAT-9 reading 

Hallinger & 
Heck, 2011 

Quantitative 

Correlational: 
Latent curve 
analysis, latent 
class analysis 

4 

West Region 


Elementary 

school 


State n = 1 
School n = 193 
Teacher n = 4,152 
Student n = 13,391 

Teacher or staff 
report 

■ Parent report 
• Student report 

Student-level, state 
test: SAT-9 math 

Heck & 

Hallinger, 2009 

Quantitative 

Correlational: 
Latent change 
analysis 

4 

West Region 


Elementary 

school 


State n = 1 
School n = 195 
Teacher n = 4,152 
Student n = 13,389 

Teacher or staff 
report 

■ Parent report 
• Student report 

Student-level, state 
test: math 

Heck & 

Hallinger, 

2010a 

Quantitative 

Correlational: 
Latent change 
analysis 

4 



Elementary 

school 


State n = 1 
School n = 195 
Student n = 13,391 

Teacher or staff 
report 

■ Parent report 
• Student report 

Student-level, state 
test: SAT-9 math 

Heck & 

Hallinger, 

2010b 

Quantitative 

Correlational: 
Latent change 
analysis 

4 

West Region 


Elementary 

school 


State n = 1 
School n = 197 
Student n = 13,391 

Teacher or staff 
report 

Parent report 
• Student report 

Student-level, state 
test: SAT-9 math and 
reading 

Heck & 
Moriyama, 
2010 

Quantitative 

Correlational: 
Structural 
equation 
modeling and 
path analysis 

4 

West Region 


Elementary 

school 


State n = 1 
School n = 198 
Student n = 25,173 

Teacher or staff 
report 

■ Parent report 
• Student report 

Student-level, state 
test: SAT-9 math and 
reading 

Horng, Klasik, 
& Loeb, 2010 

Quantitative 

Correlational: 

Regression 

Up to 1 

Miami-Dade 
County Public 
Schools, 
Florida 


Elementary, 
middle, and high 
school 


District n = 1 
School n = 65 
Principal n = 65 
Teacher n = 15,842 

Teacher or staff 
report 

■ Parent report 
Direct observation 

School-level, 
state test: Florida 
accountability model 


(continued) 
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Table Dl. Characteristics of included studies (continued) 


Study 

Methodology 

Type of analysis 

Length of 
study (years) 

Location 

Setting 

Grade level 

Participants 

Principal data source 

Student data source 

Horvat, Curci, 
& Chaplin, 
2010 

Qualitative 

Case study 

30 


Urban 

Grades K-8 

• School n = 1 

• Principal n = 3 

• Staff n = 1 

■ Teacher n = 5 

• Student n = 2 

• Parent n = 21 

• Self-report 
Teacher or staff 
report 

■ Parent report 
Direct observation 

School-level, other: 
Archival achievement 
data 

Jacobson, 
Brooks, Giles, 
Johnson, & 
Ylimaki, 2007 

Mixed method 

Correlational: 
Correlations only 
Qualitative: 

Case study 

Up to 1 

New York 

Urban 

Elementary 

school 

• School n = 3 

• Principal n = 3 

• Self-report 

• Other administrator 
report 

■ Teacher or staff 
report 

• Parent report 

• Student report 

School-level, state 
test: New York State 
Education Department 
report cards and 
reports of school 
improvement 

Kaplan, 
Owings, & 
Nunnery, 2005 

Quantitative 

Correlational: 
Analysis of 
variance 

Up to 1 

Virginia 

Mixed 

Prekindergarten, 
Elementary, 
middle, and high 
school 

■ School n = 160 

• Other administrator 
report 

Direct observation 

School-level, 
state test: Virginia 
Standards of Learning 

Knoeppel & 
Rinehart, 2008 

Quantitative 

Correlational: 
Analysis of 
covariance, 
regression 

3 

Kentucky 

Mixed 

Elementary 

school 

■ School n = 349 
• Principal n = 349 

District or state 
data 

School-level, state 
test: Commonwealth 
Accountability Testing 
System using the 
Kentucky Core 
Content Test 

Leana & Pil, 
2006 

Quantitative 

Correlational: 

Regression 

1.5 

Northeast 
& Islands 
Region 

Urban 

Elementary, 
middle, and high 
school 

• District n = 1 

• School n = 88 

• Principal n = 88 

• Teacher n = 2,167 

• Parent n = 5,130 

• Self-report 

• Other administrator 
report 

Teacher or staff 
report 

• Parent report 
Direct observation 

School-level, state 
test: English language 
arts and math 

Leithwood & 
Jantzi, 2008 

Quantitative 

Correlational: 
Structural 
equation modeling 
and path analysis 

3 

Nationwide 

Mixed 

Elementary, 
middle, and high 
school 

■ State n = 9 

• District n = 45 

■ School n = 96 

• Teacher n = 2,764 

• Self-report 
■ Teacher or staff 
report 

School-level, state 
test: English language 
arts and math 

Leithwood & 
Mascall, 2008 

Quantitative 

Correlational: 

Structural 

equation 

modeling/path 

analysis 

3 

Nationwide 

Mixed 

Elementary, 
middle, and high 
school 

■ State n = 9 

• District n = 45 

■ School n = 90 

• Teacher n = 2,570 

■ Teacher or staff 
report 

School-level, state 
test: English language 
arts and math 


(continued) 
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Table Dl. Characteristics of included studies (continued) 


Study 

Methodology 

Type of analysis 

Length of 
study (years) 

Location 

Setting 

Grade level 

Participants 

Principal data source 

Student data source 

Louis, Dretzke, 
& Wahlstrom, 
2010 

Quantitative 

Correlational: 

Structural 

equation 

modeling 

3 

Nationwide 

Mixed 

Elementary, 
middle, and high 
school 

• State n = 9 

• District n = 43 

• School n = 106 

• Teacher n = 4,491 

• Self-report 
■ Teacher or staff 
report 

School-level, state 
test: math 

Marks & Printy, 
2003 

Quantitative 

Correlational: 
Hierarchical 
linear model 

Up to 1 

Nationwide 

Mixed 

Elementary, 
middle, and high 
school 

• State n = 16 

• District n = 22 
■ School n = 24 

• Teacher n = 910 

• Self-report 

• Other administrator 
report 

Teacher or staff 

Student-level, 
other: Academic 
achievement is a 
measure of authentic 


report student performance, 

Direct observation specifically, the sum 
of averaged student 
scores in math and 
social studies on 
three standards of 
intellectual quality: 
analysis, disciplinary 
concepts, and 
elaborated written 
communication. 


Masumoto & 

Qualitative 

Case study 

Up to 1 

California 

Rural 

High school 

• District n = 3 

• Self-report 

School-level, state 

Brown-Welty, 







• School n = 3 

• Other administrator 

test: English language 

2009 







■ Superintendent 

report 

arts and math 








n = 3 

■ Teacher or staff 

School-level, other: 








• Principal n = 4 

report 

meet adequate yearly 








■ Counselor n = 2 

• Parent report 

progress; academic 








■ Advisor n = 1 

Direct observation 

performance index 








• Teacher n = 9 


scores above the state 


median, above average 
proficiency rates for 
English language arts 
and math; graduation 
rates above the state 
average, lower than 
average four-year 
dropout rates; above 
average 2004 A-G 
(course requirements 
for University of 
California admission) 
completion rates 


(continued) 
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Table Dl. Characteristics of included studies (continued) 

Study 

Methodology 

Type of analysis 

Length of 
study (years) 

Location 

Setting 

Grade level 

Participants 

Principal data source 

Student data source 

May, Huff, & 
Goldring, 2012 

Quantitative 

Correlational: 
Hierarchical 
linear model 

3 

Southeast 

Region 

Urban 

Grades 1-8 

• District n = 1 

• School n = 39 

• Principal n = 39 

• Student n = 38,510 

• Self-report: 
principal diary 

Student-level, state 
test: Achievement 
scores from the 
state assessment in 
English language arts 
and math 

O’Donnell & 
White, 2005 

Quantitative 

Correlational: 

Regression 

Up to 1 

Pennsylvania 


Grades 5-8 

• School n = 15 

• Teacher n = 250 

• Self-report 
Teacher or staff 
report 

School-level, state 
test: English language 
arts and math as 
measured by the 
Pennsylvania System 
of School Assessment 

Ovando & 
Ramirez, 2007 

Qualitative 

Case study 

Up to 1 

Texas 


Elementary, 
middle, and high 
school 

• District n = 1 

• Principal n = 3 

• Assistant principal 
n = 3 

• Self-report 
Teacher or staff 
report 

• Direct observation 

School-level, 
state test: Texas 
Assessment of 
Academic Skills 

Owings, 
Kaplan, & 
Nunnery, 2005 

Quantitative 

Correlational: 
Analysis of 
variance 

Up to 1 

Virginia 


Elementary, 
middle, and high 
school 

■ School n = 160 

• Principal n = 160 

• Superintendent 
n = 160 

• Other administrator 
report 

School-level, 
state test: Virginia 
Standards of Learning 

Reardon, 2011 

Quantitative 

Correlational: 

Regression 

Up to 1 

Virginia 

Suburban 

Elementary 

school 

■ School n = 31 
• Principal n = 31 

• Self-report 

School-level, state 
test: Virginia Standards 
of Learning in English 
language arts 

Ruff & Shoho, 
2005 

Qualitative 

Case study 

Up to 1 

San Antonio, 
Texas 

Urban 

Elementary 

school 

• School n = 3 

• Principal n = 3 

• Teacher n = 6 

• Self-report 
Teacher or staff 
report 

■ Direct observation 

• Other — document 
review 

School-level, state 
test: Texas Academic 
Excellence Indicator 
System 

Sanzo, 
Sherman, & 
Clayton, 2011 

Qualitative 

Case study 

Up to 1 

Virginia 


Grades 6-8 

• Principal n = 10 

• Self-report 

School-level, other: 
Virginia accreditation 
standards; those 
whose schools met the 
federal No Child Left 
Behind accreditation 
standards 

(continued) 
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Table Dl. Characteristics of included studies (continued) 

Study 

Methodology 

Type of analysis 

Length of 
study (years) 

Location 

Setting 

Grade level 

Participants 

Principal data source 

Student data source 

Sebastian & 
Allensworth, 
2012 

Quantitative 

Correlational: 
Structural 
equation 
modeling and 
path analysis 

Up to 1 

Chicago, 

Illinois 

Urban 

High school 

• Principal n = 1 
■ School n = 99 

• Teacher n = 3,529 

Teacher or staff 
report 

School-level, other: 
unweighted student 
grades and gains 
on the Education 
Planning and 
Assessment System. 

Sherman & 
Crum, 2007 

Qualitative 

Grounded theory 

Up to 1 

Southeast 

Region 

Suburban 

Elementary 

school 

• State n = 1 

• District n = 2 

• Principal n = 10 

• Self-report 

School-level, other: 
Schools passed state 
accountability tests in 
English language arts, 
schools nationally 
recognized for school 
improvement 

Silva, White, & 
Yoshida, 2011 

Quantitative 

Randomized 
controlled trial 

Up to 1 

Pennsylvania 

Suburban 

Grades 6-8 

• District n = 1 

■ School n = 1 

• Principal n = 1 

■ Assistant principal 
n = 2 

■ Student n = 41 

• Random 
assignment to 
condition 

Student-level, state 
test: English language 
arts as measured 
by the Pennsylvania 
System of School 
Assessment 

Supovitz, 
Sirinides, & 
May, 2010 

Quantitative 

Correlational: 

Structural 

equation 

modeling 

Up to 1 

Southeast 

Region 

Urban 

Grades 2-8 

■ District n = 1 

■ School n = 38 

• Teacher n = 721 

• Student n = 11,397 

Teacher or staff 
report 

School-level, state 
test: English language 
arts and math 

Tschannen- 
Moran, Parish, 
& DiPaola, 
2006 

Quantitative 

Correlational: 

Regression 

Up to 1 

Virginia 

Mixed 

Grades 6-8 

• School n = 82 

Teacher or staff 
report 

School-level, 
state test: Virginia 
Standards of Learning 
in English language 
arts, writing, and 
math 

Twigg, 2008 

Quantitative 

Correlational: 
Structural 
equation 
modeling and 
path analysis 

Up to 1 

Southeast 

Region 

Mixed 

Elementary, 
middle, and high 
school 

■ District n = 1 

■ School n = 31 

• Principal n = 31 
Faculty n = 363 

• Self-report 

• Other administrator 
report 

Teacher or staff 
report 

School-level, state 
test 

Valentine & 
Prater, 2011 

Quantitative 

Correlational: 
Analysis of 
variance, 
regression 

Up to 1 

Missouri 

Mixed 

High school 

■ School n = 131 

• Principal n = 131 

• Teacher n = 443 

Teacher or staff 
report 

School-level, state 
test: Missouri 
Assessment Program 
Performance Index 


(continued) 
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Table Dl. Characteristics of included studies (continued) 

Study 

Methodology 

Type of analysis 

Length of 
study (years) 

Location 

Setting 

Grade level 

Participants 

Principal data source 

Student data source 

Vanderhaar, 
Munoz, & 
Rodosky, 2006 

Quantitative 

Correlational: 
Multivariate 
analysis of 
variance, 
regression 

Up to 1 

Midwest 

Region 

Urban 

Elementary, 
middle, and high 
school 

• District n = 1 

• School n = 91 

• Principal n = 91 

District or state 
data 

School-level, 
state test: Total 
Academic Index, and 
Comprehensive Test 
of Basic Skills Total 
Battery 

Williams, 
Persaud, & 
Turner, 2008 

Quantitative 

Correlational: 
Analysis of 
variance, 
regression 

Up to 1 

Atlanta, 

Georgia 

Urban 

Elementary 

school 

• District n = 1 

• School n = 81 

Teacher or staff 
report 

School-level, state 
test: grade 4 Georgia 
Criterion Referenced 
Tests English 
language arts scores 

Williams, 2009 

Quantitative 

Correlational: 

Regression 

Up to 1 

Georgia: 

Urban 

Elementary 

school 

• District n = 1 

■ School n = 81 

• Teacher n = 3,952 

■ Teacher or staff 
report 

School-level, state 
test: grade 4 Georgia 
Criterion Referenced 
Tests English 
language arts scores 

Young, Vang, & 
Young, 2008 

Quantitative 

Correlational: 

Regression 

Up to 1 

California 

Mixed 

Elementary 

school 

• Principal n = 260 

• Self-report 
District or state 
data 

School-level, state 
test: California 
Academic 
Performance Index 

— is not available. 

Source: Authors' compilation. 




Appendix E. Summary of the findings 


The study findings of principal characteristics were organized into three categories: precun 
sors (table El), behaviors (table E2), and leadership styles (table E3). This appendix shows 
the studies in each category. 


Table El. Precursors: Conditions that shape leadership behavior and supporting 
literature 


| Precursor 

Supporting research | 

Principals' experience 

Braun, Gable, & Kite, 2011; Clark, Martorell, & Rockoff, 2009; Grissom & 
Loeb, 2011; Jacobson, Brooks, Giles, Johnson, & Ylimaki, 2007; Knoeppel & 
Rinehart, 2008; Ruff & Shoho, 2005; Vanderhaar, Munoz, & Rodosky, 2006 

Years of experience in 
education 

Knoeppel & Rinehart, 2008 

Teaching experience 
prior to becoming a 
principal 

Clark, Martorell, & Rockoff, 2009; Vanderhaar, Munoz, & Rodosky, 2006 

Years of experience as 
an assistant principal 
at current school prior 
to becoming a principal 
at the school 

Clark, Martorell, & Rockoff, 2009 

Years of experience as 
a principal 

Braun, Gable, & Kite, 2011; Grissom & Loeb, 2011; Jacobson, Brooks, Giles, 
Johnson, & Ylimaki, 2007; Knoeppel & Rinehart, 2008; Ruff& Shoho, 2005; 
Vanderhaar, Munoz, & Rodosky, 2006 

Principal preparation 
programs 

Braun, Gable, & Kite, 2011; Clark, Martorell, & Rockoff, 2009; Corcoran, 
Schwartz, & Weinstein, 2012; Donmoyer, Yennire-Donmoyer, & Galloway, 

2012; Kaplan, Owings, & Nunnery, 2005; Knoeppel & Rinehart, 2008; Owings, 
Kaplan, & Nunnery, 2005; Vanderhaar, Munoz, & Rodosky, 2006 

Types of preparation 
programs 

Clark, Martorell, & Rockoff, 2009; Corcoran, Schwartz, & Weinstein, 2012 

University- and district- 
based programs 

Vanderhaar, Munoz, & Rodosky, 2006 

Content of preparation 
programs 

Braun, Gable, & Kite, 2011; Donmoyer, Yennire-Donmoyer, & Galloway, 2012; 
Kaplan, Owings, & Nunnery, 2005; Knoeppel & Rinehart, 2008; Owings, 
Kaplan, & Nunnery, 2005 

Personal traits 
(educational attainment) 

Valentine & Prater, 2011; Young, Vang, & Young, 2008 

Source: Authors' analysis. 
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Table E2. Behaviors: Five domains of principal action and supporting literature 


Behavior Supporting research 


Instructional management 

Borko, Wolf, & Simone, 2003; Brown, Benkovitz, Muttillo, & Urban, 2011; 
Chappelear & Price, 2012; Cohen-Vogel, 2011; Crum & Sherman, 2008; 

Crum, Sherman, & Myran, 2010; Fancera & Bliss, 2011; Grissom & Loeb, 
2011; Masumoto & Brown-Welty, 2009; O’Donnell & White, 2005; Ovando & 
Ramirez, 2007; Reardon, 2011; Sanzo, Sherman, & Clayton, 2011; Sebastian 
& Allensworth, 2012; Sherman & Crum, 2007; Silva, White, & Yoshioda, 2011; 
Supovitz, Sirinides, & May, 2010; Williams, Persaud, & Turner, 2008 

Monitoring and 
providing feedback to 
teachers and students 

Chappelear & Price, 2012; Fancera & Bliss, 2011; Ovando & Ramirez, 2007; 
Silva, White, & Yoshioda, 2011; Williams, Persaud, & Turner, 2008 

Having a vision for 
learning 

Brown, Benkovitz, Muttillo, & Urban, 2011; O’Donnell & White, 2005; Reardon, 
2011; Sebastian & Allensworth, 2012 

Providing support 
and professional 
development to 
teachers 

Borko, Wolf, & Simone, 2003; Crum & Sherman, 2008; Masumoto & Brown- 
Welty, 2009; Sanzo, Sherman, & Clayton, 2011; Sherman & Crum, 2007; 
Supovitz, Sirinides, & May, 2010 

Using data to drive 
decisionmaking 

Cohen-Vogel, 2011; Crum, Sherman, & Myran, 2010; Sherman & Crum, 2007 

Internal relations 

Crum & Sherman, 2008; Egley & Jones, 2005; Gordon & Louis, 2009; Horvat, 
Curci, & Chaplin, 2010; Leana and Pil, 2006; Louis, Dretzke, & Wahlstrom, 
2010; Sanzo, Sherman, & Clayton, 2011; Sherman & Crum, 2007; Silva, White, 
& Yoshioda, 2011 

Organizational 

management 

Bloom & Owens, 2013; Cohen-Vogel, 2011; Grissom & Loeb, 2011; Horng, 
Klasik, & Loeb, 2010; May, Huff, & Goldring, 2012 

Administrative duties 

Williams, 2009 

External relations 

Gordon & Louis, 2009; Horvat, Curci, & Chaplin, 2010; Leana & Pil, 2006; 
Masumoto & Brown-Welty, 2009 


Source: Authors' analysis 


Table E3. Leadership styles and supporting literature 


| Leadership style 

Supporting research |j 

Leadership styles 

Griffith, 2004; Hallinger & Heck, 2010a, 2010b, 2011; Heck & Hallinger, 2009, 
2010a, 2010b; Heck & Moriyama 2010; Leithwood & Jantzi, 2008; Leithwood 
& Mascall, 2008; Marks & Printy, 2003; Tschannen-Moran, Parish, & DiPaola, 
2006; Twigg, 2008 

Relationship 
between distributive, 
collaborative, and 
learning-directed 
leadership and student 
achievement 

Hallinger & Heck, 2010a, 2010b, 2011; Heck & Hallinger, 2009, 2010a, 
2010b; Heck & Moriyama 2010 

Other research on 
relationship between 
leadership styles and 
student achievement 

Griffith, 2004; Leithwood & Jantzi, 2008; Leithwood & Mascall, 2008; Marks & 
Printy, 2003; Tschannen-Moran, Parish, & DiPaola, 2006; Twigg, 2008 


Leadership style Supporting research 


Source: Authors' analysis 
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Notes 


Special thanks to Stephan Cooley and Cameron Lindahl, graduate research assistants, for 

research and analytic support. Also, special thanks to Connie Verhagen and John Hughes 

for technical support. 

1. Marzano et al. (2005) identified 21 principal leadership responsibilities as having a sig- 
nificant correlation (r = .18— 33) with student achievement. 

2. Meta-analysis examining the relationships between types of school leadership (that is, 
instructional, transformational, variety of combined theories) and students’ academic 
achievement and nonacademic outcomes (for example, self-concept or participation in 
school activities) from 1978 to 2006 suggested that the effect of instructional leader- 
ship (r = .40) was nearly four times the effect of transformational leadership (r = .11; 
Robinson et al., 2008). 

3. Murphy et al. (2006) theorized the learning-centered framework. It explains that expe- 
rience, knowledge, personal characteristics, and values and beliefs are precursors to the 
principal’s leadership behaviors, which in turn, influence the overall school experience 
(for example, standards, curriculum, culture) and specific classroom experiences (for 
example, teacher practices) of the student, which affect student success (see figure B3 
in appendix B) 

4. Some studies — such as Leithwood and Jantzi (2008), Leithwood and Mascall (2008), 
Hallinger and Heck (2010a, 2010b, 2011), Heck and Hallinger (2010a, 2010b), and 
Heck and Moriyama (2010) — appear to use the same sample or subsamples. Therefore, 
the samples may not be independent, and the number of participants reported may 
overestimate the number of unique districts and schools. 

5. Precursors in this review are defined as conditions that shape principal behavior, as 
conceptualized by Murphy et al. (2006). However, the precursor categories identified in 
this review are distinct categories and are not intended to be directly aligned with the 
definitions of Murphy et al. (2006) precursor categories. 

6. The number of studies that examine particular variables are quantified by variable; 
however, in some instances authors examined more than one variable (such as teach- 
ing experience, years of experience as a principal). For example, Vanderhaar et al. 
(2006) examined both teaching experience and principal experience. Those findings 
are reflected in both applicable sections of the report, which, in some cases, impacts 
the overall counts. 

7. Effect sizes could not be calculated; Grissom and Loeb (2011, p. 1115) state, “While 
not shown, we find no relationship between principal characteristics and student per- 
formance, except for principal experience, which shows positive associations with test 
score growth in both subjects.” 

8. Findings of this study, which examined the relationships between principal quality, 
socioeconomic status of students at the school, and student achievement, revealed that 
principals in the upper two quartiles of principal quality — as measured by the Inter- 
state School Leaders Licensure Consortium principal quality rubric — led schools with 
higher student achievement. The association between principal quality and student 
achievement was significant in grades 3 (r = .48) and 5 (r = .49) but not in grade 9 or 
for end-of'high-school course exams. 

9. While Kaplan et al. (2005) and Owings et al. (2005) are two separate publications, 
identical study results are reported in both publications. 

10. Insufficient data were provided to calculate effect sizes. 
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11. Insufficient data were reported to calculate effect size so only the standardized regres- 
sion coefficient as reported as a direct effect from a structural equation model is report- 
ed here. 

12. Learning-directed leadership was characterized as distributed or collaborative leader- 
ship combined with the development of school capacity to support teaching and learn- 
ing. In turn these styles supported the capacity to improve instruction for improved 
student achievement (Heck & Moriyama, 2010). 

13. Leadership components were used for both distributed and collaborative leadership 
styles. 

14. Insufficient data were provided to calculate effect sizes so only standardized regression 
coefficients are reported here. 

15. Insufficient data were reported to calculate an effect size so only the standardized 
regression coefficient is reported here. 

16. Insufficient data were provided to calculate effect sizes so only the total effect was 
calculated and reported as a standardized regression coefficient. 
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